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Key findings 


This study examined three cohorts of Utah English learner students 
over six years. The cohorts were students in kindergarten, grade 
3, and grade 6 at the start of the study. Using scores from the 
previously administered English language proficiency assessment 
and content knowledge tests in English language arts and math, 
the study team found that at least 59 percent of English learner 
students in these cohorts were reclassified as fluent English 
proficient students over the six-year period. English learner 
students who were eligible for special education services had the 
lowest passing rates on all three tests. English learner students 
in higher grades had lower cumulative passing rates on all three 
tests than English learner students in lower grades. The overall 
cumulative passing rate was highest for the English language arts 
content test, followed by the math content test and the English 
language proficiency assessment. 
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Summary 


Before considering how to successfully educate growing numbers of English learner stu- 
dents, especially those who struggle to pass state English language arts and math content 
tests, it is important that policymakers and educators first understand the patterns of 
these students’ achievement progress. Representatives of the state office of education in 
Utah requested this study of the English language proficiency and academic progress of 
the state’s English learner students. (The state departments of education in Arizona and 
Nevada requested similar studies, which Regional Educational Laboratory West also con- 
ducted and published separately.) 

This study followed cohorts of English learner students in Utah over six school years to 
assess their progress in English proficiency and tracked their academic progress in English 
language arts and math content knowledge. It analyzed three cohorts — which started at 
kindergarten, grade 3, and grade 6 — from 2006/07 through 2011/12 by their level of English 
proficiency at the start of the study, eligibility for special education services, eligibility for 
the school lunch program (a proxy for low-income status), gender, and grade level. 

To track the students’ progress, the study used the tests Utah administered during the 
study period: the Utah Academic Language Proficiency Assessment and the state’s subject 
matter tests in English language arts and math. 1 This report describes the cumulative per- 
centage of Utah’s English learner students in each grade cohort who reached each of three 
specific milestones during the study period: meeting the criteria for reclassification as fluent 
English proficient students, passing the English language arts content test for the first time, 
and passing the math content test for the first time. The study also compared the cumu- 
lative passing rates of English learner students taking the three tests. Finally, it compared 
the students’ progress in English proficiency with Utah’s expectation that English learner 
students advance, on average, slightly less than one proficiency level each year. 

During the study period at least 59 percent of English learner students were reclassified as 
fluent English proficient students. 

The largest differences in cumulative passing rates for all three tests were associated with 
student eligibility for special education services and with students’ initial English language 
proficiency level (on a scale of 1, low, to 5, high). Smaller differences in cumulative passing 
rates were associated with student eligibility for the school lunch program and with student 
gender. 

In all three grade cohorts English learner students who started the study at the two English 
language proficiency levels just below the level needed for reclassification as fluent English 
proficient students generally had higher cumulative passing rates on all three tests than 
English learner students who started at the two lowest English language proficiency levels. 
In the grade 3 and grade 6 cohorts English learner students who started the study at the 
two lowest English language proficiency levels had final cumulative reclassification rates as 
fluent English proficient of less than 30 percent. English learner students in higher grades 
had lower cumulative passing rates on all three tests than English learner students in lower 
grades. 


Within the kindergarten cohort 77 percent of English learner students passed the English 
language arts content test, 76 percent passed the math content test, and 73 percent 
passed the Utah Academic Language Proficiency Assessment. Within the grade 3 cohort 
77 percent passed the English language arts content test, 69 percent passed the math 
content test, and 64 percent passed the Utah Academic Language Proficiency Assess- 
ment. Within the grade 6 cohort 68 percent passed the English language arts content 
test, 59 percent passed the math content test, and 59 percent passed the Utah Academic 
Language Proficiency Assessment. 

Less than 60 percent of any group of English learner students made progress in English 
fluency at the expected annual rate of slightly less than one level per year, as called for by 
Utah’s annual measurable achievement objective 1. 
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Why this study? 


Across the United States, particularly in states served by Regional Educational Laboratory 
(REL) West, concern is widespread about how to successfully educate growing numbers of 
English learner students, especially those who struggle to pass state English language arts 
and math content tests (Horwitz et al, 2009; Olsen, 2010; Quality Counts, 2009). The 
members of REL West’s English Learner Alliance, which includes representatives of state 
departments of education in Arizona, Nevada, and Utah, requested studies of the English 
language proficiency and academic progress of English learner students in their states. This 
study is for Utah. 2 

Having a better understanding of the progress of English learner students in both English 
language proficiency and subject matter knowledge will enable English Learner Alliance 
members to more effectively target interventions for students who are not achieving 
English language proficiency within expected timeframes and for those not passing English 
language arts and math content tests. 

While previous studies examined some of these issues, the study durations were generally 
shorter than in this study (box 1). Few, if any, studies directly examined the progress of cohorts 
of English learner students over five or more years, and none examined English learner stu- 
dents’ progress on subject matter tests in English language arts and math, based both on 
students’ initial English language proficiency level and on their initial grade level. This report 
addresses this gap in the literature by providing empirical evidence on the progress of grade 
cohorts of English learner students in English language proficiency and in English language 
arts and math content knowledge over several years. It also examined how these outcomes 
differed by student subgroups. See box 2 for definitions of key terms used in the report. 


Box 1. Previous studies show English learner students tend to lag behind native 
English speakers on academic achievement tests 

English learner students, as a group, tend to lag behind native English speakers in their rate of 
academic achievement (Kindler, 2002; Massachusetts Department of Elementary and Second- 
ary Education, 2012; Olsen, 2010; Ruiz-de-Velasco & Fix, 2000; Short & Fitzsimmons, 2007). 
This gap largely reflects English learner students’ need to simultaneously learn English and 
master subject matter knowledge (Genesee, Lindholm-Leary, Saunders, & Christian, 2005). 
Flowever, English learner students are a diverse group with different strengths and needs, 
depending on a number of characteristics (Kindler, 2002). 

Characteristics that appear to be related to academic achievement for English learner stu- 
dents specifically, and for students generally, for which most states and districts collect data 
include initial English language proficiency when students first enroll in school (Cook, Linquanti, 
Chinen, & Jung, 2012; Collier, 1989, 1992; Halle, Flair, Wandner, McNamara, & Chien, 2012). 
They also include grade level (Genesee et al., 2005); poverty status (Goldenberg, 2008; Mul- 
ligan, Halle, & Kinukawa, 2012; Rathbun & West, 2004; Roberts, 2009; Roberts & Bryant, 
2011); disability status (Liasidou, 2013; McCardle, McCarthy-Mele, Cutting, Leos, & D’Emilio, 
2005; Nguyen, 2012); and gender (Perie, Moran, & Lutkus, 2005). 

Initial English language proficiency and grade level. Research shows that English learner stu- 
dents at the same English language proficiency level tend to make greater year-to-year progress 
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Box 1. Previous studies show English learner students tend to lag behind native 
English speakers on academic achievement tests (continued) 

in English language proficiency and subject matter knowledge in the lower grades than they do 
in the higher grades (Cook, Wilmes, Boals, & Santos, 2008; Grissom, 2004; Kieffer, 2008, 
2010, 2011; Salazar, 2007). 

Poverty status. English learner students from homes of lower socioeconomic status generally 
score lower on subject matter tests and are less likely to achieve reclassification as fluent 
English proficient students than their peers of higher socioeconomic status (Mulligan et al., 
2012; Roberts & Bryant, 2011). 

English learner students with disabilities. Nearly 400,000 English learner students in 
the United States in grades K-12 were identified as needing special education services in 
2001/02 (McCardle et al., 2005). While a learning disability can affect a student’s academ- 
ic achievement, it is often difficult to determine whether English learner students struggle to 
develop literacy and other academic benchmarks because of their limited English proficiency 
or because they have a learning disability (Klingner, Artiles, & Barletta, 2006; Nguyen, 2012). 

Gender. Differences in academic achievement by gender have been found among K-12 stu- 
dents, including small but persistent gender disparities in math favoring male students 
(McGraw, Lubienski, & Strutchens, 2006; Perie et al., 2005) and small gender disparities in 
reading favoring female students (Perie et al., 2005). 


Box 2. Key terms 

Annual measurable achievement objective 1. In annual measurable achievement objective 1 
the No Child Left Behind Act of 2001 asked states to set expectations for how quickly English 
learner students should progress from one English proficiency level to the next, measured by 
annual increases in the number or percentage of students making progress in learning English. 
Utah, like about half the states, set an expectation of advancing an average of less than one 
English language proficiency level per school year. Utah defines annual measurable achieve- 
ment objective 1 progress differently depending on the English learner student’s English lan- 
guage proficiency level. To determine adequate progress on annual measurable achievement 
objective 1, beginning (level 2) and developing (level 3) are divided into two sublevels, A and 
B, creating the following annual minimum progress levels: entering (level 1), beginning A (level 
2A), beginning B (level 2B), developing A (level 3A), developing B (level 3B), expanding (level 
4), and bridging (level 5). Utah expects English learner students to score at least at the next 
higher English proficiency progress level. At Utah’s average annual measurable achievement 
objective 1 progress rate of slightly less than one English language proficiency level per school 
year English learner students would be expected to be reclassified as fluent English proficient 
students within one to six years of the start of the study, depending on the level at which they 
started in 2006/07. See appendix A. 

Cohorts. Students were grouped into three analytic cohorts based on their grade level in 
2006/07: kindergarten, grade 3, and grade 6. Within each cohort students’ initial English 
language proficiency level was determined based on the 2006/07 Utah Academic Language 
Proficiency Assessment. The first subject matter tests were also administered in 2006/07. The 
study covered six years: 2006/07-2011/12. Thus the kindergarten cohort followed students 
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Box 2. Key terms (continued) 


from kindergarten to grade 5, the grade 3 cohort from grade 3 to grade 8, and the grade 6 
cohort from grade 6 to grade 11 (see also box 3). 

English language proficiency levels. During the study period Utah had two different labels for 
the five levels of English language proficiency. Prior to 2010/11 the five proficiency levels were 
pre-emergent (level 1), emergent (level 2), intermediate (level 3), advanced (level 4), and fluent 
(level 5). In 2010/11 the five proficiency levels were renamed entering (level 1), beginning 
(level 2), developing (level 3), expanding (level 4), and bridging (level 5). The two versions of 
the proficiency levels are equivalent to each other, and the exit standards are intended to be 
the same. For clarity the current proficiency level labels are used in this study. The levels are 
based on the Utah Academic Language Proficiency Assessment (see below). Throughout this 
report English language proficiency level refers to the English language proficiency level in the 
first year of the study period. 

English learner students. Students are classified as English learner students if they fall into 
levels 1-4 on the Utah Academic Language Proficiency Assessment. Students are asked to 
take the assessment when they initially register as new students if their family speaks a lan- 
guage other than English at home. 

Fluent English proficient students. Before 2010/11 the Utah State Office of Education used 
both the Utah Academic Language Proficiency Assessment and the English language arts 
content test (described below) to identify English learner students’ level of English language 
proficiency and to determine whether they would be reclassified as fluent English proficient. 
To achieve reclassification English learner students had to achieve an overall Utah Academic 
Language Proficiency Assessment score of level 4 (currently named expanding) or above and 
an English language arts content test score of partial (level 2 of 4) or above, which is one level 
below passing (sufficient, level 3 of 4). Starting in 2010/11 English learner students were 
reclassified as fluent English proficient when they achieved level 5 (currently named bridging) 
on the Utah Academic Language Proficiency Assessment. 

Special education services. All special education services or individualized education pro- 
grams under this Utah State Office of Education designation in the state dataset were included 
in this study sample. Data were not collected on individual types of learning disabilities or 
special education services within this general category. 

Utah Academic Language Proficiency Assessment. This assessment makes the initial deter- 
mination of whether a student is classified as an English learner and places the student 
at one of five levels of English proficiency. The test measures proficiency in four domains: 
listening, speaking, reading, and writing. As described above (see fluent English proficient 
students), Utah used two different standards to determine reclassification as fluent English 
proficient: one before 2010/11 and another starting in 2010/11. During the study period 
Utah administered the Utah Academic Language Proficiency Assessment each spring. After 
the conclusion of the study period, Utah switched to WIDA ACCESS as its English language 
proficiency test. 

Utah’s English language arts and math content tests. Utah administers content tests in 
several subjects, including English language arts and math. The English language arts content 
test is grade specific for both the elementary and secondary grades. The math content test 
is grade specific for the elementary grades and course specific (for example, Algebra 1 and 
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Box 2. Key terms (continued) 


Geometry) for the secondary grades. Prior to 2008 Utah administered its content tests in 
grades 1-11. In spring 2008 Utah began administering its content tests in grades 2-11, and 
beginning in spring 2011 Utah administered its content tests in grades 3-11. For the study 
period, which began in 2006/07, only the kindergarten cohort was affected by these changes 
to the content test administration. This study described student progress on the content tests 
in the kindergarten cohort beginning in grade 2 in spring 2009. The content tests have four 
performance levels: minimal (level 1), partial (level 2), sufficient (level 3), and substantial (level 
4). Students must score at or above sufficient to pass the English language arts and math 
content tests. During the study period Utah administered its content tests each spring. Utah 
no longer administers the content tests used during the study period, having replaced them 
with its own Student Assessment of Growth and Excellence content tests in English language 
arts and math in 2014/15. 


What the study examined 


This study is a descriptive analysis of the progress of three grade cohorts of Utah English 
learner students in English language proficiency and in English language arts and math 
content knowledge over 2006/07-2011/12. Student English language proficiency scores and 
English language arts and math content test scores were followed over the study period. 

Three cohorts of students were examined based on their grade level in 2006/07: kindergar- 
ten, grade 3, and grade 6. Students’ initial English language proficiency level was based on 
results on the 2006/07 Utah Academic Language Proficiency Assessment 3 (see box 2). 

The study examined the cumulative percentages of students in each of these cohorts 
who reached each of three specific academic milestones over the course of the study 
(2006/07-2011/12): 

• Scoring at or above the level for reclassification as fluent English proficient stu- 
dents on the Utah Academic Language Proficiency Assessment. 

• Passing the English language arts content test for the first time. 

• Passing the math content test for the first time. 

The study also examined how meeting these criteria varied by students’ initial English Ian- 
guage proficiency level (see box 2), eligibility for special education services, eligibility for 
the school lunch program (a proxy for low-income status), gender, and grade level (which, 
of course, changed over the study period). 

First, to determine how many and how quickly English learner students became proficient 
in English, the study looked at cumulative reclassification rates: 

• What was the cumulative percentage of English learner students from each cohort 
who were reclassified as fluent English proficient students after the baseline year 
(2006/07)? 

• How did the cumulative percentage of English learner students who achieved 
reclassification as fluent English proficient students vary by students’ initial English 
language proficiency level, eligibility for special education services, eligibility for 
the school lunch program, gender, and grade level? 
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Second, to determine how well English learner students did academically in English Ian- 
guage arts and math, the study looked at cumulative passing rates: 

• What was the cumulative percentage of English learner students from each cohort 
who passed the English language arts and math content tests for the first time 
after the baseline year? 

• How did the cumulative percentage of English learner students who passed the 
English language arts and math content tests for the first time vary by students’ 
initial English language proficiency level, eligibility for special education services, 
eligibility for the school lunch program, gender, and grade level? 


The study also compared the cumulative reclassification rates of English learner students 
on the Utah Academic Language Proficiency Assessment and the cumulative passing rates 
on the English language arts and math content tests. Finally, it compared the progress 
rates in English language proficiency for English learner students at each English language 
proficiency level with Utah’s progress expectations on annual measurable achievement 
objective 1, of advancing, on average, slightly less than one English language proficiency 
level each year. 

Box 3 summarizes the study’s data sources and methods. Appendix B provides more detail. 


Box 3. Data and methods 

Data source The state of Utah provided data on all students who had English language pro- 
ficiency test results and subject matter test results in English language arts and math from 
2006/07 through 2011/12, starting in kindergarten, grade 3, and grade 6. This dataset 
enabled Regional Educational Laboratory West to examine aspects of these students’ prog- 
ress in English language proficiency and subject matter knowledge over six school years. 
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Analysis sample and methods. Because the study analyzed the entire population of Utah 
English learner students who met the analytic sample criteria for each of the grade cohorts, 
statistical tests were not conducted. 

The analytic sample included all students identified as English learner students who were 
enrolled in the state’s public schools in the designated grade of the first year of the cohort, 
who progressed to the next grade level each year, and for whom the required test data were 
available for all six years being analyzed. 

Each cohort consisted of a separate sample of students. For example, the students in the 
grade 3 cohort were English learner students who enrolled in a Utah public school in grade 3 
in 2006/07, progressed to the next grade level each year, and for whom the required Utah test 
score data were available through grade 8 in 2011/12. 

Each cohort was progressively smaller because younger students who met the reclassi- 
fication criteria as fluent English proficient students in the earlier grades were not included 
in the grade 3 and grade 6 cohorts. Further, the number of newly registering English learner 
students in grade 3 in the grade 3 cohort and in grade 6 in the grade 6 cohort was smaller 
than the number of English learner students who reclassified as fluent English proficient in the 
earlier grade levels. For example, for the Utah Academic Language Proficiency Assessment, 
the kindergarten cohort had 3,237 students, the grade 3 cohort had 2,088 students, and the 
grade 6 cohort had 1,152 students. For a description of the steps taken to define each analyt- 
ic sample see table B1 in appendix B. 
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Box 3. Data and methods (continued) 


The analyses were done for each English learner grade cohort as a whole and by both 
cohort and each of four student characteristics at the start of 2006/07: English language pro- 
ficiency level, eligibility for special education services, eligibility for the school lunch program, 
and gender. For a breakdown of the characteristics of the whole sample and each cohort see 
table B2 in appendix B. 

For the English language proficiency analysis 2007/08 was the first year data were avail- 
able on measured progress (relative to 2006/07), and 2011/12 was the final year, for a total 
of five years of progress measurement. For the subject matter tests English learner student 
achievement levels for 2006/07-2011/12 were examined, for a total of six years. 

See appendix B for further details on data and methods. 


What the study found 


All three cohorts of English learner students made moderate progress in achieving reclas- 
sification as fluent English proficient students, with 59-73 percent of each cohort’s students 
achieving reclassification as fluent English proficient students based on their performance 
on the Utah Academic Language Proficiency Assessment (and, before 2010/11, on the 
English language arts content test as well; see box 2). English learner students in the kin- 
dergarten cohort had the highest cumulative reclassification rate, which was consistent 
with previous research. 

As expected, English learner students who started the study at expanding proficiency 
(level 4), the level just below that needed for reclassification as fluent English proficient stu- 
dents, had the highest cumulative reclassification rates. However, in the grade 3 and grade 
6 cohorts, English learner students who started the study at the lowest English language 
proficiency level (entering) had cumulative reclassification rates that were higher than or 
similar to those of students who began the study at level 2 (beginning). 

Cumulative passing rates on the subject matter tests — Utah’s English language arts and 
math content tests — were similar for both subjects. English learner students in the kinder- 
garten cohort had the highest cumulative passing rates in English language arts and math, 
while English learner students in the grade 6 cohort had the lowest cumulative passing 
rates. This finding is consistent with previous research that found that English language 
students in lower grades made better academic progress than students in higher grades. For 
all three grade cohorts English learner students who started the study at the two highest 
English language proficiency levels below that needed for reclassification as fluent English 
proficient students (developing, level 3, and expanding, level 4) had the highest cumulative 
passing rates in English language arts and math, as expected. However, in the grade 3 
and grade 6 cohorts English learner students who started the study at the lowest English 
language proficiency level (entering) had cumulative reclassification rates in English lan- 
guage arts that were higher than or similar to students who began the study at level 2 
(beginning). 

Across all three tests the cumulative passing rate for the three grade cohorts was highest 
for the English language arts content test (68-77 percent), followed by the math content 
test (59-76 percent), and then by the Utah Academic Language Proficiency Assessment 
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(59-73 percent). The largest differences in cumulative passing rates for all three tests were 
associated with students’ eligibility for special education services and with their English 
language proficiency level at the start of the study. 

English learner students eligible for the school lunch program scored lower on the Utah 
Academic Language Proficiency Assessment and the English language arts content test 
than their peers who were not eligible for the school lunch program, and male students 
scored lower than female students. Female English learner students scored lower than their 
male peers on the math content test. 

Less than 60 percent of any group of English learner students made the expected progress of 
slightly less than one English proficiency level per year called for in Utah’s annual measurable 
achievement objective 1. At the same time more than two-thirds of English learner students 
eventually reclassified in 7 of the 15 proficiency level groups across the three grade cohorts. 4 

Specific results related to the research questions are presented below. 

How many English learner students became proficient in English during the five-year study period 
and how quickly 

Less than three-fourths of English learner students were reclassified as fluent English 
proficient, and students made consistent progress during the study period. Across all 
three grade cohorts between 59 percent (grade 6 cohort) and 73 percent (kindergarten 
cohort) of English learner students scored at or above the required level for reclassifica- 
tion as fluent English proficient on the Utah Academic Language Proficiency Assessment 
(figure 1). In the grade 3 cohort 64 percent of the English learner students reclassified. 
Overall, English learner students made consistent progress in cumulative reclassification 
rates during the study period. 

English learner students who started the study at the highest two English proficien- 
cy levels below that needed for reclassification as fluent English proficient students 
scored highest on the Utah Academic Language Proficiency Assessment. Compared 
with English learner students who began the study at lower English language proficiency 
levels, English learner students at expanding proficiency (level 4) and developing proficien- 
cy (level 3) had the two highest cumulative reclassification rates on the Utah Academic 
Language Proficiency Assessment (figure 2). In the kindergarten cohort English learner 
students at beginning proficiency (level 2) and entering proficiency (level 1) had the two 
lowest cumulative reclassification rates. In the grade 3 and grade 6 cohorts English learner 
students who started the study at entering proficiency (level 1) had slightly higher final 
cumulative passing rates as fluent English proficient than the English learner students who 
started the study at beginning proficiency (level 2). 

English learner students who were eligible for special education services had lower 
cumulative passing rates on the Utah Academic Language Proficiency Assessment than 
their peers who were not eligible. For all three grade cohorts English learner students who 
were eligible for special education services at the start of the study had lower cumulative 
reclassification rates as fluent English proficient than students who were not eligible (figure 
3). After five years this gap remained consistent in the kindergarten cohort and increased 
in the grade 3 and grade 6 cohorts. The kindergarten cohort had the highest cumulative 
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Figure 1. For all three grade cohorts of English learner students the cumulative 
reclassification rate in becoming fluent English proficient followed a steady 
progression from the beginning to the end of the study period, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. Prior to 2010/11 the Utah State Office of Education 
used both the Utah Academic Language Proficiency Assessment and the English language arts content test 
to reclassify English learner students; beginning in 2010/11 only the Utah Academic Language Proficiency As- 
sessment was used. For the kindergarten cohort of English learner students reclassification started in grade 2 
since there was no English language arts content test for grade 1 in 2007/08. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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reclassification rate for English learner students eligible for special education services at the 
start of the study, at 52 percent. 

Cumulative reclassification rates were lower for English learner students who were 
eligible for the school lunch program than for their peers who were not eligible and were 
lower for male English learner students than for their female counterparts. The differ- 
ence in cumulative reclassification rates between English learner students who were eligi- 
ble for the school lunch program and those who were not ranged from 6 percentage points 
in the grade 6 cohort to 7 percentage points in the kindergarten cohort (see figure Cl in 
appendix C). The difference between male and female English learner students ranged 
from 2 percentage points in the grade 6 cohort to 8 percentage points in the kindergarten 
cohort (see figure C2 in appendix C). 

How well English learner students did academically in English language arts and math 

At least 68 percent of English learner students passed the English language arts content 
test during the six-year study period. The cumulative passing rate on the English language 
arts content test ranged from 68 percent in the grade 6 cohort to 77 percent in both the kin- 
dergarten and grade 3 cohorts (figure 4). Similar to the cumulative reclassification rate progress 
on the Utah Academic Language Proficiency Assessment, the cumulative passing rate prog- 
ress on the English language arts content test was generally consistent throughout the study. 
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Figure 2. For the grade 3 and grade 6 cohorts English learner students at the two 
lowest initial English language proficiency levels made the slowest cumulative 
reclassification rate progress in becoming fluent English proficient, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. Prior to 2010/11 the Utah State Office of Education 
used both the Utah Academic Language Proficiency Assessment and the English language arts content test 
to reclassify English learner students; beginning in 2010/11 only the Utah Academic Language Proficiency As- 
sessment was used. For the kindergarten cohort of English learner students reclassification started in grade 2 
since there was no English language arts content test for grade 1 in 2007/08. The number of English learner 
students in each subgroup is as follows. Kindergarten cohort: entering, 267; beginning, 805; developing, 
1,225; and expanding, 940. Grade 3 cohort: entering, 69; beginning, 224; developing, 1,715; and expanding, 
80. Grade 6 cohort: entering, 39; beginning, 76; developing, 940; and expanding, 97. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 


English learner students who began the study at higher English language proficien- 
cy levels generally had higher cumulative passing rates on the English language arts 
content test than students who started at lower proficiency levels. In all three grade 
cohorts English learner students who started the study at the two highest English Ian- 
guage proficiency levels below the level needed for reclassification (expanding, level 4, 
and developing, level 3) had the highest cumulative passing rates on the English language 
arts content test (figure 5). In the grade 3 and grade 6 cohorts English learner students 
who started the study at expanding proficiency (level 4) had the highest final cumula- 
tive passing rates in their cohorts despite the fact that none of those students passed the 
English language arts content test in the first year of the study. In the kindergarten cohort 
English learner students at beginning proficiency (level 2) had a higher cumulative passing 
rate than English learner students at entering proficiency (level 1). However, for the grade 
3 and grade 6 cohorts English learner students who started the study at the entering level 
(level 1) had higher cumulative passing rates on the English language arts content test 
than their peers who started the study at beginning proficiency (level 2). Of the students in 
the grade 3 and grade 6 cohorts who began the study at the two lowest English language 
proficiency levels, 50 percent or less passed the English language arts content test. 
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Figure 3. The difference in cumulative reclassification rates as fluent English 
proficient students between English learner students who were eligible for special 
education services and those who were not was largest in the grade 6 cohort, 
followed by the grade 3 cohort, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. Prior to 2010/11 the Utah State Office of Education 
used both the Utah Academic Language Proficiency Assessment and the English language arts content test to 
reclassify English learner students; beginning in 2010/11 only the Utah Academic Language Proficiency Assess- 
ment was used. For the kindergarten cohort of English learner students reclassification started in grade 2 since 
there was no English language arts content test for grade 1 in 2007/08. The number of English learner students 
in each subgroup is as follows. Kindergarten cohort: eligible for special education, 192; not eligible for special 
education, 3,045. Grade 3 cohort: eligible for special education, 366; not eligible for special education, 1,722. 
Grade 6 cohort: eligible for special education, 288; not eligible for special education, 864. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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English learner students who were eligible for special education services at the start of 
the study had lower cumulative passing rates on the English language arts content test 
than students who were not eligible. The difference in cumulative passing rates decreased 
for the kindergarten cohort and increased for both the grade 3 and grade 6 cohorts as the 
study progressed (figure 6). The gap ranged from 14 percentage points in the kindergarten 
cohort to 28 percentage points in the grade 6 cohort. In the grade 6 cohort more than half 
of English learner students who were eligible for special education services at the begin- 
ning of the study never passed the English language arts content test. 

Cumulative passing rates in English language arts were lower for English learner stu- 
dents who were eligible for the school lunch program than for their peers who were 
not eligible and were lower for male English learner students than for their female 
peers. English language students who were eligible for the school lunch program had lower 
cumulative passing rates on the English language arts content test than their peers who 
were not eligible. The difference ranged from 4 percentage points in the kindergarten 
cohort to 8 percentage points in the grade 3 cohort (see figure C3 in appendix C). Male 
English learner students had lower cumulative passing rates on the English language arts 
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Figure 4. For the kindergarten and grade 3 cohorts the cumulative passing 
rate progress on the English language arts content test was greater than the 
cumulative passing rate progress for the grade 6 cohort, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the 
initial population of English learner students for this analysis. In Utah students take the English language arts 
content test in grades 2-11. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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content test than their female peers. The gap ranged from 5 percentage points in the grade 
6 cohort to 6 percentage points in the kindergarten and grade 3 cohorts (see figure C4). 

Cumulative passing rates on the math content test showed both similar and different 
patterns among subgroups and cohorts compared with rates on the English language 
arts content test: the kindergarten cohort scored highest; however, progress eventually 
slowed for the grade 3 and grade 6 cohorts. Across the three grade cohorts the cumula- 
tive passing rate on the math content test ranged from 59 percent in the grade 6 cohort to 
76 percent in the kindergarten cohort (figure 7). Different from the progress on the English 
language arts content test, the cumulative passing rate progress in the grade 3 and grade 6 
cohorts on the math content test was greater during the first few years of the study period 
and then slowed toward the end of the study. This change was most evident in the grade 6 
cohort, where the passing rate increased from year 1 (grade 6) to year 2 (grade 7) and then 
slowed by year 3 (grade 8) to a nearly flat rate. 

English learner students who began the study at higher English language proficiency 
levels had higher cumulative passing rates on the math content test than students who 
started at lower proficiency levels. The patterns of cumulative passing rate progress for the 
math content test were more consistent than those for the English language arts content test. 
In all three grade cohorts English learner students at each English language proficiency level 
had higher cumulative passing rates than their peers at lower English language proficiency 
levels (figure 8). For the grade 3 and grade 6 cohorts the difference in cumulative passing 
rates between English learner students who began the study at higher and lower English 
proficiency levels increased over the course of the six-year study period. For example, in the 
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Figure 5. For the grade 3 and grade 6 cohorts English learner students at the two 
lowest initial English language proficiency levels had the slowest progress rates on 
the English language arts content test, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. In Utah students take the English language arts content 
test in grades 2-11. The number of English learner students in each subgroup is as follows. Kindergarten cohort: 
entering, 246; beginning, 809; developing, 1,247; and expanding, 1,051. Grade 3 cohort: entering, 29; beginning, 
200; developing, 1,719; and expanding, 78. Grade 6 cohort: entering, 14; beginning, 44; developing, 755; and 
expanding, 70. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 


grade 3 cohort the gap in the math passing rate between English learner students at expand- 
ing (level 4) and entering (level 1) increased from 13 percentage points in year 1 of the study 
(grade 3) to 47 percentage points in year 6 of the study (grade 8). However, the final cumu- 
lative differences were small between some groups. The gap was 4 percentage points or less 
between the English learner students at entering proficiency (level 1) and those at beginning 
proficiency (level 2) in the kindergarten and grade 3 cohorts and between developing profi- 
ciency (level 3) and expanding proficiency (level 4) in the grade 6 cohort. 
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The highest cumulative passing rate on the math content test (91 percent) was achieved 
by English learner students in the kindergarten cohort who began the study at expanding 
(level 4), the highest English language proficiency level below that needed for reclassifi- 
cation as fluent English proficient. English learner students with the lowest cumulative 
passing rates on the math content test (50 percent) were those in the grade 3 and grade 
6 cohorts who began the study at entering proficiency (level 1) and beginning proficiency 
(level 2), the two lowest English language proficiency levels. 


English learner students who were eligible for special education services at the start 
of the study had lower cumulative passing rates on the math content test than stu- 
dents who were not eligible, and the gap widened toward the end of the study for the 
grade 3 and grade 6 cohorts. Like the results on the English language arts content test, 
English learner students who were eligible for special education services at the beginning 
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Figure 6. English learner students who were eligible for special education 
services started with lower passing rates on the English language arts content 
test than students who were not eligible, a difference that decreased over time 
for the kindergarten cohort and increased for the grade 3 and grade 6 cohorts, 
2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. In Utah students take the English language arts content 
test in grades 2-11. The number of English learner students in each subgroup is as follows. Kindergarten cohort: 
eligible for special education, 192; not eligible for special education, 3,161. Grade 3 cohort: eligible for special 
education, 350; not eligible for special education, 1,676. Grade 6 cohort: eligible for special education, 235; not 
eligible for special education, 648. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 


of the study had lower cumulative passing rates on the math content test than their peers 
who were not eligible for special education services (figure 9). In the grade 3 and grade 

6 cohorts the gap in cumulative passing rates increased during the six-year study period. 
The gap increased 3 percentage points at the end of the study for the grade 3 cohort and 
4 percentage points for the grade 6 cohort. For the kindergarten cohort the gap decreased 

7 percentage points. The kindergarten cohort had the highest pass rate for English learner 
students who were eligible for special education services (67 percent), and the grade 6 
cohort had the lowest (47 percent). 

English learner students eligible for the school lunch program and female English 
learner students had lower cumulative passing rates on the math content test than 
their peers who were not eligible and than male English learner students. English 
learner students who were eligible for the school lunch program had lower cumulative 
passing rates on the math content test than their peers who were not eligible. The dif- 
ference ranged from 4 percentage points in the kindergarten and grade 6 cohorts to 
6 percentage points in the grade 3 cohort (see figure C5 in appendix C). Female English 
learner students had lower cumulative passing rates on the math content test than their 
male English learner peers. Female and male English learner students had about the same 
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Figure 7. For English learner students in the grade 6 cohort cumulative passing 
rate progress on the math content test slowed after the first two years of the study 
period, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. In Utah students take grade-specific math content tests 
in elementary school and course-specific math content tests in secondary school. Students take content tests in 
grades 3-8 and in either grade 10 or 11 in high school; there is no test in grade 9. 

Source: Authors' analysis of student- 1 eve I data from the Utah State Office of Education, 2006/07-2011/12. 
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cumulative passing rate on the math content test in the kindergarten cohort, while the 
difference was 2 percentage points in the grade 3 cohort, and 9 percentage points in the 
grade 6 cohort (see figure C6). 

Comparison of cumulative passing rates across tests and cohorts 

With one exception English learner students across all cohorts scored higher on the 
subject matter tests than on the Utah Academic Language Proficiency Assessment. Across 
all three tests the overall cumulative passing rate for each of the three grade cohorts was 
highest for the English language arts content test, followed by the math content test, and 
then the Utah Academic Language Proficiency Assessment (table 1). The one exception 
occurred in the grade 6 cohort, where English learner students had the same cumula- 
tive passing rate on the math content test and the Utah Academic Language Proficiency 
Assessment (59 percent). 

Comparison of cumulative passing rates with Utah’s progress expectations 

Utah’s annual measurable achievement objective 1. Utah’s annual measurable achieve- 
ment objective 1 for English learners students — an average increase of slightly less than 
one English language proficiency level per school year — means that English learner stu- 
dents should have passed the Utah Academic Language Proficiency Assessment within 
one to six years of the start of the study, depending on the proficiency level at which 
they started in 2006/07. For example, English learner students who started the study at 
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Figure 8. By the end of the study the gap in cumulative passing rates on the math 
content test increased between the two highest and the two lowest initial English 
language proficiency levels for the grade 3 and grade 6 cohorts, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 served as a baseline, defining the initial 
population of English learner students for this analysis. In Utah students take grade-specific math content tests 
in elementary school and course-specific math content tests in secondary school. The number of English learner 
students in each subgroup is as follows. Kindergarten cohort: entering, 245; beginning, 810; developing, 1,247; 
and expanding, 1,051. Grade 3 cohort: entering, 41; beginning, 198; developing, 1,674; and expanding, 73. Grade 
6 cohort: entering, 33; beginning, 61; developing, 972; and expanding, 95. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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beginning proficiency (level 2) should have been able to achieve reclassification as fluent 
English proficient students within five years of the start of the study. 

English learner students did not generally achieve the expected rate of progress in 
English proficiency but rather advanced at different rates. How well English learner 
students in this sample met the progress expectation differed according to their cohort 
and language proficiency level at the start of the study. However, less than half of the 
English learner students in all but one of the cohort English language proficiency level 
groups achieved reclassification as fluent English proficient within the expected timeline of 
advancing at an average rate of slightly less than one English language proficiency level per 
school year (see table A1 in appendix A). The proportion of English learner students who 
met the expected reclassification timeline ranged from 7 percent (grade 3 cohort students 
who started the study at developing A, level 3A) to 58 percent (kindergarten cohort stu- 
dents who started at beginning proficiency, level 2). 


15 


Figure 9. The difference in cumulative passing rates in math between English 
learner students who were eligible for special education services and those who 
were not decreased over time for the kindergarten cohort and increased for the 
grade 3 and grade 6 cohorts, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 served as a baseline, defining the initial 
population of English learner students for this analysis. In Utah students take grade-specific math content tests 
in elementary school and course-specific math content tests in secondary school. The number of English learner 
students in each subgroup is as follows. Kindergarten cohort: eligible for special education, 193; not eligible for 
special education, 3,160. Grade 3 cohort: eligible for special education, 321; not eligible for special education, 
1,665. Grade 6 cohort: eligible for special education, 275; not eligible for special education, 886. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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Table 1. Cumulative passing rates for each cohort for each test, 2006/07-2011/12 


(percent of students passing the test) 

Test 

Kindergarten 

Grade 3 

Grade 6 

Utah Academic Language Proficiency Assessment 

73 

64 

59 

English language arts content test 

77 

77 

68 

Math content test 

76 

69 

59 


Note: Number of English learner students in each analytic sample is as follows. Kindergarten cohort: English 
language proficiency, 3,237; English language arts, 3,353; and math, 3,353. Grade 3 cohort: English language 
proficiency, 2,088; English language arts, 2,026; and math, 1,986. Grade 6 cohort: English language profi- 
ciency, 1,152; English language arts, 883; and math, 1,161. 

Source: Authors' analysis of student- 1 eve I data from the Utah State Office of Education, 2006/07-2011/12. 
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Implications of the study findings 


This section includes some implications of the study findings for policy, practice, and 
future research and discusses how the findings expand on or vary from the findings of 
previous research. 

Three findings are consistent with current research and suggest areas for focused interventions 

Utah may want to consider devoting additional attention to improving teaching prac- 
tices and support services to help the following underperforming English learner student 
subgroups. 

English learner students who are eligible for special education services. All English 
learner students who are eligible for special education services will likely need addition- 
al support to be successful, and this support may need to vary by specific subgroups of 
learning disabilities. Further investigation into more effective practices for improving the 
achievement of English learner students who are eligible for special education services, 
especially English learner students in secondary school, appears warranted. 

Older English learner students. English learner students in the grade 6 cohort always 
made less progress than the lower cohorts on all three tests. Older English learner students 
may require additional, possibly different, supports if they are to meet at least minimal 
expectations for academic achievement. Or, perhaps, middle and high school teachers will 
need additional or different skills from those they currently have. These teachers may need 
additional, targeted professional development to effectively support the academic English 
literacy needs of their older English learner students across content areas. 

English learner students eligible for the school lunch program and male and female 
English learner students, depending on the subject. While performance differences 
between English learner students who were eligible for the school lunch program and 
those were not and between male and female English learner students were generally 
smaller than those for the other two characteristics examined (eligibility for special educa- 
tion services and initial English language proficiency level), they were consistently present. 
Accordingly, comprehensive supports to improve English learner student achievement 
would likely need to include supports targeted to the additional or unique needs of English 
learner students who are eligible for the school lunch program. Similarly, additional or 
targeted supports for reclassification as fluent English proficient students and in English 
language arts for male English learner students and in math for female English learner 
students may be warranted. 

Four findings expand the current research literature and point to areas for possible changes in 
practice or further research 

Four of the study findings offer evidence not found in the general research literature. 

Differentiated support by students’ English language proficiency levels may be needed to 
help students at the lowest English language proficiency levels succeed on current and 
future subject matter tests. For this study the achievement standard on the subject matter 
tests was low: passing the content test once over the course of six years. Nevertheless, 
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English learner students who started the study at lower proficiency levels still had low 
cumulative passing rates on English language arts and math content tests, especially com- 
pared with their gradedevel peers who started the study at higher English language pro- 
ficiency levels. For example, the gap in cumulative passing rates on the English language 
arts content test in the grade 3 cohort was 46 percentage points between English learner 
students who started the study at expanding proficiency, level 4 (91 percent), and those 
who started the study at beginning proficiency, level 2 (45 percent). While these find- 
ings are consistent with the research literature (Cook et al., 2008; Grissom, 2004; Kieffer, 
2008, 2010, 2011; Salazar, 2007), they also show something more: that large percentages 
of English learner students who were at the lower English language proficiency levels were 
not able to advance sufficiently in English proficiency and subject matter knowledge to 
pass subject matter tests once over six years. These findings suggest that changes in teach- 
ing and support practices may be needed if English learners at lower English language 
proficiency levels are ever going to meet minimum content achievement expectations. 

Examining English learner student achievement among special education students by 
specific learning disabilities may provide useful information for more effective, targeted 
supports. In this study English learner students who were eligible for special education 
services had lower final cumulative passing rates on all three tests than their peers who 
were not eligible for special education services, which is consistent with the research lit- 
erature (see, for example, Lipka, Siegel, & Vukovic, 2005). Additional research could take 
into consideration the fact that English learner students who are eligible for special edu- 
cation services cover a diverse set of learning-related disabilities — ranging from autism to 
hearing impairments to emotional disabilities to traumatic brain injury (Kavale, 2005). 
Examining the achievement rates of these different groups may provide useful information 
on how to best support each. 

Utah may want to examine the linguistic complexity of its English language proficiency 
and subject matter tests. English learner students had higher cumulative passing rates on 
the subject matter tests than on the Utah Academic Language Proficiency Assessment. 
Since no research was found that directly compared English learner students’ performance 
across multiple years on these three types of assessments, this study adds useful research. 
Based on the expected level of linguistic complexity, English learner students should pass 
the Utah Academic Language Proficiency Assessment most often, followed by the math 
content tests and then the English language arts content tests (see, for example, Abedi & 
Dietel, 2004; Abedi & Gandara, 2006; Abedi & Lord, 2001). It was surprising to find that 
the opposite occurred. Perhaps these results indicate that the level of academic English lit- 
eracy necessary for success on the English language proficiency and the math and English 
language arts content tests are different from what was expected. Further research is 
needed in this area if expected levels of achievement across these three tests are going to 
align with actual results. 

Utah may need to consider changing its annual measurable achievement objectives or 
its teaching and support practices, as most English learner students did not meet the 
progress and achievement targets. This study’s findings show that across grade cohorts 
and English language proficiency levels, less than 50 percent of the English learner stu- 
dents in all but one of the cohort English proficiency-level subgroups made the expected 
rate of progress. The percentage of English learner students who met the expected rate of 
progress ranged widely from 7 percent (grade 3 cohort, developing A proficiency, level 3A) 


Based on the 
expected level 
of linguistic 
complexity, English 
learner students 
should pass the 
Utah Academic 
Language 
Proficiency 
Assessment most 
often, followed by 
the math content 
tests and then the 
English language 
arts content tests. 
It was surprising 
to find that the 
opposite occurred 
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to 58 percent (kindergarten cohort, beginning proficiency, level 2). At the same time the 
percentage of English learner students within any of the 15 cohort English language profi- 
ciency level subgroups who eventually reclassified during the five years of the study period 
ranged from 22 percent (grade 6 cohort, entering proficiency, level 1) to 92 percent (kin- 
dergarten cohort, expanding proficiency, level 4). Thus, it may be too rigid or simplistic to 
assume that all English learner students, regardless of English language proficiency level 
and grade level, will advance at the same pace. More flexible standards may be useful to 
educators. Further, that such low percentages of English learner students meet the expect- 
ed achievement targets may also signal that teaching and support practices are not effec- 
tive. Improvements in these areas may also increase English learner student achievement. 

One finding was not consistent with the research literature and points to areas for additional 
research 

Utah may want to examine why some groups of English learner students outperformed 
their grade-level peers with higher English language proficiency levels. This study finding 
was not consistent with the general research literature and points to areas for additional 
research to better understand the discrepancies between this study’s results and the current 
research base. 5 

In a few instances English learner students with lower initial English language proficiency 
levels had higher cumulative passing rates than their peers who had higher initial pro- 
ficiency levels. In the grade 3 and grade 6 cohorts English learner students at entering 
proficiency (level 1) had cumulative passing rates on the Utah Academic Language Pro- 
ficiency Assessment and English language arts content test that were comparable to or 
slightly higher than their peers at beginning proficiency (level 2). This did not occur in the 
kindergarten cohort on any of the three tests, nor did it occur on the math content test in 
any of the three grade cohorts. It is not clear why this difference occurred. One possibility 
is that this finding may be driven by variation in the prevalence of risk factors, such as 
eligibility for special education services or the school lunch program, across students with 
different English language proficiency levels. Or, perhaps, the actual difference in English 
proficiency between English learner students at these two lower levels is much smaller than 
between any other levels, especially in the higher grade levels. Further analyses in these 
areas could help clarify this inconsistency with the research literature. 

Limitations of the study 
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This study has two limitations. The first relates to the scope of the sample. The analytic 
sample excludes mobile students who left or entered Utah public schools during the study 
period. It also excludes students who repeated or skipped a grade, because of difficulties 
tracking students who did not progress with the rest of their grade cohort. As a result, this 
sample is a more stable group of English learner students than is the case in most schools. 
Thus, the cumulative passing rates on examined tests could be higher than for the English 
learner population as a whole. To address the possibility that the study sample may differ 
from the English learner population as a whole, table B2 in appendix B describes the dif- 
ference between the whole population and the analytic sample. 

The second limitation relates to comparisons among grade cohorts based on differences 
in sample characteristics. The students in the kindergarten, grade 3, and grade 6 cohorts 
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likely have different characteristics, especially related to initial English language proficien- 
cy level. For students in the kindergarten cohort, kindergarten was their initial enrollment 
year in Utah public schools, and their English language proficiency level was their initial 
English language proficiency level when they started school. For students in the grade 3 
and grade 6 cohorts their English language proficiency level was their level at the start of 
the study, which most likely differed from their proficiency level at the time they started 
school. Furthermore, the composition of the kindergarten, grade 3, and grade 6 cohorts 
could vary because of differences in student mobility, screening opportunities for special 
education services, and grade repetition across cohorts. 


20 



Appendix A. Utah programs that provide context for the study 


To provide context for the analysis in this report, this appendix describes Utah’s process 
for the identification of students who are eligible for special education services, the idem 
tification of students as English learner students, the Utah Academic Language Proficien- 
cy Assessment and proficiency levels, English language arts and math content tests and 
achievement levels, English learner support programs, and how Utah (and other states) 
defines “making progress in learning English.” These descriptions provide a context for the 
state analysis and are not intended as evaluations of the state programs or assessments. 

Identification of students who are eligible for special education services 

Students are eligible for special education services if they are determined to have a learn- 
ing disability under the Individuals with Disabilities Education Act of 2004 (34 C.F.R. 
Secs. 300 et al.) and Section 504 of the Rehabilitation Act of 1973. Utah’s procedures for 
making this determination are intended to comply with these federal statutes, Utah state 
statutes, and the Utah administrative code. The procedures are described in the Utah 
State Office of Education’s (2008) Specific Learning Disabilities Eligibility Guidelines. The 
Utah State Office of Education formed a focus group composed of education specialists on 
its staff, representatives from local education agencies, parents, disability law center staff, 
related service providers, and Mountain Plains Regional Resource Center staff to create 
the guidelines for a comprehensive evaluation of whether a student has a learning disabili- 
ty or impairment and is eligible to receive special education services. The focus group iden- 
tified three methods that local education agencies may use in making this determination: 

• Student’s response to increasing levels of response to intervention activities 
(method A). 

• Level of discrepancy between the student’s achievement and intellectual ability 
(method B). 

• Combination of data from the response to intervention and discrepancy methods 
(method C). 

Regardless of which eligibility determination method is used, the process must include all 
of these elements of the comprehensive evaluation: 

• Creation of an evaluation team that includes parents and qualified professionals. 

• Observation of the student’s learning environment by someone other than the stu- 
dent’s teacher, and repeated assessments at reasonable intervals, to determine that 
° The student does not make adequate achievement for his or her age. 

° The student’s inadequate achievement is not due to lack of appropriate 
instruction in reading and math. 

• Demonstration that the basis for the learning disability and eligibility determina- 
tion is in line with one of the three methods. 

• Certification, in writing, by each evaluation team member that the report reflects 
his or her conclusion. 

If a student is determined to be eligible for special education services, the specially designed 
instruction for the student is set forth in the student’s individualized education program. 

In Utah, as in all states, each local education agency must make proactive efforts to identi- 
fy, locate, and evaluate students with disabilities within its jurisdiction. 
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Identification of students for an English learner program 


As required by Title III of the No Child Left Behind Act of 2001, each state must distribute 
a home language survey to all students when they first enroll in the state’s public schools. 
The schools must assess the English language proficiency of all students whose parents or 
guardians report that a language other than English is spoken at home. Utah administers 
its own version of a home language survey and uses the Utah Academic Language Profi- 
ciency Assessment test to make the initial determination of whether a student should be 
classified as an English learner student. Students whose parents or guardians report that a 
language other than English is spoken at home and who do not pass the Utah Academic 
Language Proficiency Assessment are classified as English learner students. 

English language proficiency test and subject matter tests 

Utah administers the Utah Academic Language Proficiency Assessment each spring. It 
was developed to test students in five grade spans (K-l, 2-3, 4-5, 6-8, and 9-12), with 
each grade span testing four different domains: listening, writing, reading, and speaking. 
Prior to 2010/11, the five proficiency levels were pre-emergent, emergent, intermediate, 
advanced, and fluent. During this time the Utah State Office of Education used both the 
Utah Academic Language Proficiency Assessment and the English language arts content 
test to identify English learner students’ level of English language fluency and to determine 
whether they would achieve reclassification. To achieve reclassification as fluent English 
proficient, English learner students had to achieve an overall Utah Academic Language 
Proficiency Assessment score of advanced or above and an English language arts content 
test score of partial (level 2 of 4) or above, which is one level below passing (sufficient, level 
3 of 4). As of 2010/11 the five proficiency levels were renamed entering (formerly pre-emer- 
gent), beginning (formerly emergent), developing (formerly intermediate), expanding (for- 
merly advanced), and bridging (formerly fluent). Since 2010/11 the Utah State Office of 
Education has used only the Utah Academic Language Proficiency Assessment results 
to reclassify English learner students. A student who achieves bridging proficiency (level 
5) reclassifies as fluent English proficient and exits the English learner program. The two 
versions of the proficiency levels are equivalent to each other, and the exit standards are 
intended to be the same. For clarity, the current proficiency level labels are used in this 
study. 

Utah administers the English language arts and math content tests each spring. The 
English language arts content test is grade specific for both the elementary and secondary 
grades. The math content test is grade specific for the elementary grades and course specif- 
ic (for example, Algebra 1 and Geometry) for the secondary grades. Each content test has 
four proficiency levels: minimal (level 1), partial (level 2), sufficient (level 3), and substan- 
tial (level 4). Students must score at or above sufficient to pass the English language arts 
and math content tests. Prior to 2008 Utah administered its subject matter tests in grades 
1—11. In spring 2008 Utah began administering its subject matter tests in grades 2—11, and 
beginning in spring 2011 Utah administered its subject matter tests in grades 3—11. For the 
study period, which began in 2006/07, only the kindergarten cohort was affected by these 
changes to the subject matter test administration. This study described student progress on 
the subject matter tests in the kindergarten cohort beginning in grade 2 in spring 2009. 
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Types of English learner support programs 


In Utah English learner support programs vary by district. Across districts English learner 
support programs include one-way and two-way immersion programs, English as a second 
language programs, and newcomer programs that address the language and cultural tran- 
sition needs of recent-immigrant students, among others. English learner support programs 
also include strategies and resources for mainstream teachers of English learner students 
— most specifically through content-based English learner programs, such as the Specially 
Designed Academic Instruction in English program (Utah State Office of Education Title 
III Program, 2014). 

Guidelines for making progress in learning English 

States have discretion to determine what is considered “making progress in learning 
English” under the annual measurable achievement objective 1 requirement of Title III of 
the No Child Left Behind Act of 2001. According to an American Institutes for Research 
brief prepared for the U.S. Department of Education in May 2010, half of the states with 
sufficient documentation of their classification criteria (17 of 34 states examined in the 
study) defined annual measurable achievement objective 1 progress as advancing one 
English language proficiency level (or more) per school year until students score at the 
required English language proficiency level for reclassification as fluent English proficient 
(Boyle, Taylor, Hurlburt, & Soga, 2010). Utah is similar to the other half of the 34 states, 
which defined progress as various rates of less than one English language proficiency level 
per school year. 

Utah defines annual measurable achievement objective 1 progress differently depending 
on the English learner student’s English language proficiency level (Utah State Office of 
Education, 2014). For annual measurable achievement objective 1 purposes, beginning pro- 
ficiency (level 2) and developing proficiency (level 3) are divided into two levels, A and 
B, creating the following annual minimum progress levels: entering proficiency (level 1), 
beginning A proficiency (level 2A), beginning B proficiency (level 2B), developing A profi- 
ciency (level 3A), developing B proficiency (level 3B), expanding proficiency (level 4), and 
bridging proficiency (level 5). 

Thus, at Utah’s annual measurable achievement objective 1 rate of progress — an average 
of slightly less than one English language proficiency level per school year — Utah English 
learner students would be expected to achieve reclassification as fluent English proficient 
in one to six years, depending on their initial English language proficiency level. At the 
faster rate of one English language proficiency level per school year, English learner stu- 
dents would be expected to achieve reclassification as fluent English proficient within one 
to four years, depending on their initial English language proficiency level. Table A1 com- 
pares the percentage of students who were reclassified within Utah’s expected annual mea- 
surable achievement objective 1 timeline and those who reclassified over the longer course 
of the study period for each English language proficiency level. For example, for the English 
learner students who started at developing B proficiency (level 3B) in the kindergarten 
cohort, 41 percent were reclassified within two years as expected by annual measurable 
achievement objective 1, while 79 percent passed within the five years of the study period 
for the Utah Academic Language Proficiency Assessment. 
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Table Al. Percentage of English learner students achieving reclassification as fluent English proficient 
within the expected numbers of years and by actual number of years, by initial English language 
proficiency level, 2006/07-2011/12 


English language 
proficiency level 
in 2006/07 

AMAO 1 


Kindergarten cohort 
(percent) 

Grade 3 cohort 
(percent) 

Grade 6 cohort 
(percent) 

Expected 
years to 
reclassification 
as fluent English 
proficient student 

Target 

year 

Cumulative 
passing 
rate by 
AMA01 
expected 
year 

Actual 
cumulative 
passing 
rates at 
the end of 
the study 

Cumulative 
passing 
rate by 
AMA01 
expected 
year 

Actual 
cumulative 
passing 
rates at 
the end of 
the study 

Cumulative 
passing 
rate by 
AMA01 
expected 
year 

Actual 
cumulative 
passing 
rates at 
the end of 
the study 

Entering (level 1) 

6 

2011/12 

na 

43 

na 

30 

na 

23 

Beginning (level 2) a 

5 

2010/11 

58 

58 

26 

26 

22 

22 

Developing A (level 3A) 

3 

2009/10 

44 

69 

7 

41 

9 

27 

Developing B (level 3B) 

2 

2008/09 

41 

79 

22 

79 

18 

68 

Expanding (level 4) 

1 

2007/08 

na 

92 

28 

88 

17 

71 


AMAO 1 is annual measurable achievement objective 1, which sets an expectation of an average increase of slightly less than one 
English language proficiency level per school year for English learner students, na is not applicable. 

Note: Prior to 2010/11 the Utah State Office of Education used both the English language proficiency assessment and the English lan- 
guage arts content test to reclassify English learner students; beginning in 2010/11 only the English language proficiency assessment 
was used. The English learner students in the kindergarten cohort started to be reclassified from grade 2 since there was no English 
language arts content test for grade 1 in 2007/08. 

a. There were no cutscores to determine level 2A (beginning A) and level 2B (beginning B) in 2006/07. Thus, in this table, it is assumed 
that the students at level 2 all started from level 2A. So the results are overestimated, with all level 2 students having five years to 
reach reclassification as fluent English proficient. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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Appendix B. Data and methodology 


This appendix describes construction of the analytic samples and explains how the data 
were analyzed. 

Analytic sample 

Students were grouped into three analytic cohorts based on their grade level in 2006/07: 
kindergarten, grade 3, and grade 6. Within each grade cohort students’ initial English Ian- 
guage proficiency level was determined based on the 2006/07 Utah Academic Language 
Proficiency Assessment and, until 2010/11, Utah’s English language arts content test (box 
Bl). Thus, 2006/07 was the English learner baseline identification year, and English learner 
student proficiency progress was initially measured in 2007/08. For the English language 
arts and math content tests achievement progress for English learner students was measured 
from the first year of the study, 2006/07, with the exception of the students in the kinder- 
garten cohort. Those students were first administered the English language arts and math 
content tests in grade 2, which was 2008/09 for the students in the kindergarten cohort. 

For each grade cohort the analytic sample was based on the following criteria (students 
were included in the analytic sample if they met criteria 1-3 below, as well as either 4A, 
4B, or 4C): 

1. Was in the data system in all six years, 2006/07-2011/12. 

2. Had an initial English language proficiency-level assessment score lower than profi- 
cient in 2006/07. 

3. Started from the cohort grade (K, 3, or 6) in 2006/07 and had normal grade progress 
(no grade repeaters or grade skippers) through 2011/12. 


Box Bl. Timing of Utah Academic Language Proficiency Assessment and English 
language arts and math content tests 

The analytic period differed for the Utah Academic Language Proficiency Assessment and the 
subject matter tests. For the English language proficiency analysis 2006/07 was the baseline 
year for the identification of the English language proficiency level subgroups. That year was 
chosen because there were no English language proficiency level data prior to 2006/07 across 
the three states, Arizona, Nevada, and Utah, which were required for parallel reports that were 
part of this analysis. English language proficiency progress was measured from that point. 
Thus, in the English language proficiency analysis 2007/08 was the first year of measured 
progress, and 2011/12 was the final year, for a total of five years. For the subject matter 
tests students that were classified as English learner students in 2006/07 were identified, 
and for the grade 3 and grade 6 cohorts, their achievement levels from 2006/07 through 
2011/12 were examined, for a total of six years. For students in the kindergarten cohort who 
were classified as English learner students in 2006/07, their first subject matter tests were 
administered in grade 2, which was in 2008/09. Subject matter achievement levels for the 
English learner students in the kindergarten cohort were examined for a total of four years, 
from 2008/09 through 2011/12. 
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and 


4A. For the English language proficiency-level assessment analysis achieved English Ian- 
guage proficiency assessment level for reclassification as fluent English proficient or 
took the Utah Academic Language Proficiency Assessment in the last school year 
( 2011 / 12 ). 

or 

4B. For the English language arts analysis, had state English language arts content test 
results in each year the test was administered during the study period. 

or 

4C. For the math analysis, had state math content results in each year the test was admin- 
istered during the study period. 

In sum, the analytic sample included all students identified as English learner students 
who were enrolled in Utah public schools in the designated grade of the first year of the 
cohort, who progressed to the next grade level in each year of the study, and for whom the 
required test data were available for all six years being analyzed. Each cohort consisted of a 
separate sample of students. For example, the students in the grade 3 cohort were English 
learner students who enrolled in Utah public schools in grade 3 in 2006/07, progressed to 
the next grade level each year, and for whom the required Utah test score data were avail- 
able through grade 8 in 2011/12. 

Because of these criteria for inclusion, the sample excluded mobile students who left and 
entered each district during the study period. Grade repeaters or skippers were exclud- 
ed because tests differ by grade level. Thus, it is not accurate to annually aggregate test 
results across a cohort of students when students are taking different grade-level tests, such 
as a second grade repeater and a third grader in the same year. Further, it is difficult to 
track students who did not progress with the rest of their grade cohort, which would have 
required districts providing additional years of data for only a small percentage of students. 

The numbers and percentages for English learner students who did not make normal 
grade progress were as follows: kindergarten cohort, 77 (1.6 percent); grade 3 cohort, 20 
(0.7 percent); and grade 6 cohort, 23 (1.2 percent; table Bl). 

Hence, because the final sample was a more geographically stable population, as well as 
one without grade repeaters and skippers, the proficiency rates and passing rates could be 
higher than for the English learner population as a whole. Limitations because of the char- 
acteristics of the analytic sample and other issues are described above and in the limita- 
tions section of the main report. 

The steps for preparing the student samples for each of the three assessments (English 
proficiency, English language arts content, and math content) are described in table Bl. 
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Table Bl. Steps to develop the analytic sample for each English learner student assessment, 


2006/07-2011/12 










Sample for Utah 
Academic Language 
Proficiency 
Assessment 

Sample for English 
language arts 
content test 

Sample for math 
content test 

Cohort Step 

Sample category 

Number 

Percent 

Number 

Percent 

Number 

Percent 1 

Kindergarten Start point 

English learner students 
in 2006/07 (initial English 
language proficiency level < 5) 

4,678 

100.0 

4,678 

100.0 

4,678 

100.0 

Step 1 

Students excluded because 

they were not in the data 
system all six years 

1,015 

21.7 

1,015 

21.7 

1,015 

21.7 

Step 2 

Students excluded because 








of abnormal grade progress 

77 

1.6 

77 

1.6 

77 

1.6 

Step 3 

Students excluded because 








of missing values 

349 

7.5 

233 

5.0 

233 

5.0 

End point 

Analytic sample 

3,237 

69.2 

3,353 

71.7 

3,353 

71.7 

Grade 3 Start point 

English learner students 
in 2006/07 (initial English 
language proficiency level < 5) 

3,048 

100.0 

3,048 

100.0 

3,048 

100.0 

Step 1 

Students excluded because 

they were not in the data 
system all six years 

648 

21.3 

648 

21.3 

648 

21.3 

Step 2 

Students excluded because 








of abnormal grade progress 

20 

0.7 

20 

0.7 

20 

0.7 

Step 3 

Students excluded because 








of missing values 

292 

9.6 

354 

11.6 

394 

12.9 

End point 

Analytic sample 

2,088 

68.5 

2,026 

66.5 

1,986 

65.2 

Grade 6 Start point 

English learner students 
in 2006/07 (initial English 
language proficiency level < 5) 

1,969 

100.0 

1,969 

100.0 

1,969 

100.0 

Step 1 

Students excluded because 

they were not in the data 
system all six years 

490 

24.9 

490 

24.9 

490 

24.9 

Step 2 

Students excluded because 








of abnormal grade progress 

23 

1.2 

23 

1.2 

23 

1.2 

Step 3 

Students excluded because 








of missing values 

304 

15.4 

573 

29.1 

295 

15.0 

End point 

Analytic sample 

1,152 

58.5 

883 

44.8 

1,161 

59.0 

Note: Percentages might not sum to 100 because of rounding. 






Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 




Data analysis 

The data include student-level data for 2006/07-2011/12. Data were analyzed in the three 
parallel six-year grade-span cohorts: kindergarten-grade 5, grades 3-8, and grades 6-11. 
Annual cumulative numbers and percentages of English learner students who met each 
progress criterion were calculated and grouped by grade cohort (an analytical method 
recommended by Cook et al., 2012). At the start of the study (2006/07) analyses were 
conducted for each English learner grade cohort as a whole, as well as by four student 
characteristics: English language proficiency level, eligibility for special education services, 
eligibility for the school lunch program, and gender. The similarities and differences across 
the three cohorts were also explored. 
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Note that Utah administers its state subject matter tests starting from grade 2. Therefore, 
the kindergarten cohort has results only for grades 2-5. 

Characteristics of students in the sample and the cohorts 

Table B2 compares the characteristics of students in the analytic samples with the entire 
initial English learner student population and all analytic samples. 


Table B2. Characteristics of students in the analytic samples and the entire initial English learner 
population, 2006/07 

Grade 
cohort and 

Sample for 

Utah Academic 
Language 
Proficiency 
Assessment 

Sample for English 
language arts 
content test 

Sample for math 
content test 

Initial English 
learner population 
in 2006/07 

characteristic 

Student characteristic Number Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Kindergarten 

Gender 


Female 1,542 47.6 

1,627 

48.5 

1,632 

48.7 

2,239 

47.9 


Male 1,695 52.4 

1,726 

51.5 

1,721 

51.3 

2,439 

52.1 


Eligibility for the school lunch program in 2006/07 


Eligible 1,913 59.1 

1,996 

59.5 

1,996 

59.5 

2,813 

60.1 


Not eligible 1,324 40.9 

1,357 

40.5 

1,357 

40.5 

1,865 

39.9 


Eligibility for special education services in 2006/07 


Eligible 192 5.9 

192 

5.7 

193 

5.8 

286 

6.1 


Not eligible 3,045 94.1 

3,161 

94.3 

3,160 

94.2 

4,392 

93.9 


Initial English language proficiency level in 2006/07 


Entering 267 8.2 

246 

7.3 

245 

7.3 

440 

9.4 


Beginning 805 24.9 

809 

24.1 

810 

24.2 

1,153 

24.6 


Developing 1,225 37.8 

1,247 

37.2 

1,247 

37.2 

1,697 

36.3 


Expanding 940 29.0 

1,051 

31.3 

1,051 

31.3 

1,388 

29.7 


Total number of students 3,237 

3,353 


3,353 


4,678 


Grade 3 

Gender 


Female 946 45.3 

923 

45.6 

898 

45.2 

1,382 

45.3 


Male 1,142 54.7 

1,103 

54.4 

1,088 

54.8 

1,666 

54.7 


Eligibility for the school lunch program in 2006/07 


Eligible 1,652 79.1 

1,585 

78.2 

1,552 

78.1 

2,396 

78.6 


Not eligible 436 20.9 

441 

21.8 

434 

21.9 

652 

21.4 


Eligibility for special education services in 2006/07 


Eligible 366 17.5 

350 

17.3 

321 

16.2 

544 

17.8 


Not eligible 1,722 82.5 

1,676 

82.7 

1,665 

83.8 

2,504 

82.2 


Initial English language proficiency level in 2006/07 


Entering 69 3.3 

29 

1.4 

41 

2.1 

130 

4.3 


Beginning 224 10.7 

200 

9.9 

198 

10.0 

356 

11.7 


Developing 1,715 82.1 

1,719 

84.8 

1,674 

84.3 

2,429 

79.7 


Expanding 80 3.8 

78 

3.8 

73 

3.7 

133 

4.4 


Total number of students 2,088 

2,026 


1,986 


3,048 



(continued) 
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Table B2. Characteristics of students in the analytic samples and the entire initial English learner 
population, 2006/07 (continued) 


Grade 
cohort and 


Sample for 

Utah Academic 
Language 
Proficiency 
Assessment 

Sample for English 
language arts 
content test 

Sample for math 
content test 

Initial English 
learner population 
in 2006/07 

characteristic 

Student characteristic 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Number 

Percent 

Grade 6 

Gender 


Female 

501 

43.5 

362 

41.0 

506 

43.6 

875 

44.4 


Male 

651 

56.5 

521 

59.0 

655 

56.4 

1,094 

55.6 


Eligibility for the school lunch program in 2006/07 


Eligible 

912 

79.2 

673 

76.2 

900 

77.5 

1,546 

78.5 


Not eligible 

240 

20.8 

210 

23.8 

261 

22.5 

423 

21.5 


Eligibility for special education services in 2006/07 


Eligible 

288 

25.0 

235 

26.6 

275 

23.7 

487 

24.7 


Not eligible 

864 

75.0 

648 

73.4 

886 

76.3 

1,482 

75.3 


Initial English language proficiency level in 2006/07 


Entering 

39 

3.4 

14 

1.6 

33 

2.8 

105 

5.3 


Beginning 

76 

6.6 

44 

5.0 

61 

5.3 

163 

8.3 


Developing 

940 

81.6 

755 

85.5 

972 

83.7 

1,537 

78.1 


Expanding 

97 

8.4 

70 

7.9 

95 

8.2 

164 

8.3 


Total number of students 

1,152 


883 


1,161 


1,969 



Note: Percentages might not sum to 100 because of rounding. 

Source: Authors’ analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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Appendix C. Additional findings 


This appendix presents additional findings related to English learner students’ cumulative 
reclassification rates as fluent English proficient and cumulative passing rates on Utah’s 
English language arts content test and math content test based on eligibility for the school 
lunch program (a proxy for low-income status) and gender. 

English language proficiency 

English learner students who were eligible for the school lunch program at the start of 
the study had lower cumulative reclassification rates as fluent English proficient stu- 
dents than their peers who were not eligible. English learner students who were eligible 
for the school lunch program had lower final cumulative reclassification rates as fluent 
English proficient (that is, passing the Utah Academic Language Proficiency Assess- 
ment) than their peers who were not eligible. The difference was 7 percentage points 
or less in each of the three grade cohorts (figure Cl). For example, in the kindergarten 


Figure Cl. For the kindergarten cohort the difference in progress in achieving 
reclassification as fluent English proficient narrowed over the course of the study 
between English learner students who were eligible for the school lunch program 
and those who were not, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. Prior to 2010/11 the Utah State Office of Education 
used both the Utah Academic Language Proficiency Assessment and the English language arts content test 
to reclassify English learner students; beginning in 2010/11 only the Utah Academic Language Proficiency As- 
sessment was used. For the kindergarten cohort of English learner students reclassification started in grade 2 
since there was no English language arts content test for grade 1 in 2007/08. The number of English learner 
students in each subgroup is as follows. Kindergarten cohort: eligible for the school lunch program, 1,913; 
not eligible for the school lunch program, 1,324. Grade 3 cohort: eligible for the school lunch program, 1,652; 
not eligible for the school lunch program, 436. Grade 6 cohort: eligible for the school lunch program, 912; not 
eligible for the school lunch program, 240. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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cohort, which had the highest cumulative reclassification rate as fluent English proficient 
students for this characteristic, 70 percent of English learner students who were eligible 
for the school lunch program achieved reclassification as fluent English proficient, and 
77 percent of English learner students who were not eligible achieved reclassification. For 
the grade 3 and grade 6 cohorts English learner students who were eligible for the school 
lunch program at the start of the study had similar cumulative reclassification rates as 
English learner students who were not eligible. However, after five years there was a small 
difference. 

For the three grade cohorts female English learner students had higher cumulative 
reclassification rates as fluent English proficient than male English learner students, 
though after five years the differences were 8 percentage points or less. Across all three 
grade cohorts female English learner students had higher cumulative reclassification rates 
as fluent English proficient than male English learner students; however, the differenc- 
es between the subgroups in the final cumulative reclassification rates were 8 percentage 
points or less (figure C2). For example, for the grade 6 cohort 61 percent of female English 
learner students achieved reclassification as fluent English proficient, while 58 percent of 
male English learner students did — a difference of 3 percentage points. The difference in 
the final cumulative reclassification rate between female and male English learner students 


Figure C2. Across all three grade cohorts female English learner students 
consistently achieved higher cumulative reclassification rates as fluent English 
proficient students than did male English learner students, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. Prior to 2010/11 the Utah State Office of Education 
used both the Utah Academic Language Proficiency Assessment and the English language arts content test 
to reclassify English learner students; beginning in 2010/11 only the Utah Academic Language Proficiency As- 
sessment was used. For the kindergarten cohort of English learner students reclassification started in grade 2 
since there was no English language arts content test for grade 1 in 2007/08. The number of English learner 
students in each subgroup is as follows. Kindergarten cohort: female, 1,542; male, 1,695. Grade 3 cohort: 
female, 946; male, 1,142. Grade 6 cohort: female, 501; male, 651. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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as fluent English proficient was greatest in the kindergarten cohort, at 8 percentage points. 
For the grade 3 cohort the difference was 4 percentage points. 

English language arts content test 

For the three grade cohorts English learner students who were eligible for the school 
lunch program at the start of the study had lower cumulative passing rates on Utah’s 
English language arts content test than did English learner students who were not eligi- 
ble, a difference that remained after six years. English learner students who were eligible 
for the school lunch program had lower cumulative passing rates on the English language 
arts content test than their grade peers who were not eligible for the school lunch program 
(figure C3). For example, in the kindergarten cohort the English learner students who were 
eligible for the school lunch program had a final cumulative passing rate of 75 percent on 
the English language arts content test, while their peers who were not eligible had a final 
cumulative passing rate of 79 percent on the English language arts content test. For the 
kindergarten cohort the difference in the final cumulative passing rate between the two 
groups was 4 percentage points, while the initial difference — based on results from the first 
English language arts content test given to the kindergarten cohort students during grade 
2 — was 7 percentage points. 


Figure C3. English learner students who were not eligible for the school lunch 
program consistently had higher cumulative passing rates on the English language 
arts content test than did students who were eligible, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. In Utah students take the English language arts con- 
tent test in grades 2-11. The number of English learner students in each subgroup is as follows. Kindergarten 
cohort: eligible for the school lunch program, 1,996; not eligible for the school lunch program, 1,357. 

Grade 3 cohort: eligible for the school lunch program, 1,585; not eligible for the school lunch program, 441. 
Grade 6 cohort: eligible for the school lunch program, 673; not eligible for the school lunch program, 210. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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For the three grade cohorts female English learner students had higher cumulative 
passing rates on the English language arts content test than did male English learner 
students; after six years the differences remained similar for the kindergarten cohort 
and increased for the grade 3 and grade 6 cohorts. Across all three grade cohorts female 
English learner students had higher cumulative passing rates on the English language arts 
content test than did male English learner students (figure C4). For example, for the grade 
3 cohort female English learner students had a final cumulative passing rate of 80 percent, 
while the male English learner students had a final cumulative passing rate of 74 percent, 
a difference of 6 percentage points. The initial difference was 4 percentage points. For the 
grade 6 cohort the difference between female and male students grew between the initial 
passing rate (2 percentage points higher for female students) and the third-year (grade 8) 
cumulative passing rate (13 percentage points higher for female students), before dropping 
back to 5 percentage points higher for female students for the final cumulative passing rate. 

Math content test 

English learner students who were eligible for the school lunch program had lower 
cumulative passing rates on Utah’s math content test than did students who were not 
eligible , a difference that remained constant over time for the kindergarten cohort but 
not for the grade 3 and grade 6 cohorts. English learner students who were eligible for the 
school lunch program had lower cumulative passing rates on the math content test than 
did their gradedevel peers who were not eligible (figure C5). For example, in the grade 3 


Figure C4. Female English learner students consistently had higher English language 
arts passing rates than did male English learner students, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. In Utah students take the English language arts con- 
tent test in grades 2-11. The number of English learner students in each subgroup is as follows. Kindergarten 
cohort: female, 1,627; male, 1,726. Grade 3 cohort: female, 923; male, 1,103. Grade 6 cohort: female, 362; 
male, 521. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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cohort, English learner students who were eligible for the school lunch program had a 
cumulative passing rate of 68 percent on the math content test, while their peers who were 
not eligible for the school lunch program had a cumulative passing rate of 74 percent, a 
difference of 6 percentage points. The difference in the math passing rate at the start of 
the study for the grade 3 cohort was 3 percentage points. The differences ranged from 4 
percentage points in the kindergarten and grade 6 cohorts to 6 percentage points in the 
grade 3 cohort. 

Across all three cohorts the difference in cumulative passing rates on the math content 
test for female English learner students and male English learner students ranged from 
0 to 9 percentage points. The difference between male and female students in cumulative 
passing rates on the math content test was never greater than 9 percentage points each 
year. That 9 percentage point difference occurred twice in the grade 6 cohort: first during 
the grade 6 cohort’s year 3 (grade 8), with a cumulative passing rate of 51 percent for female 
English learner students and a 60 percent rate for male English learner students, and again 
in the final year (grade 11), with a cumulative passing rate of 54 percent for female English 
learner students and a 63 percent rate for male English learner students (figure C6). In 
the grade 3 cohort female English learner students had a final cumulative passing rate of 
68 percent on the math content test, while the male English learner students had a final 


Figure C5. English learner students who were eligible for the school lunch program 
had lower cumulative passing rates on the math content test than did students who 
were not eligible, a difference that remained constant over time for the kindergarten 
cohort but not for the grade 3 and grade 6 cohorts, 2006/07-2011/12 
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Note: Students' English language proficiency assessment in 2006/07 serves as a baseline, defining the initial 
population of English learner students for this analysis. In Utah students take grade-specific math content tests 
in elementary school and course-specific math content tests in secondary school. The number of English learner 
students in each subgroup is as follows. Kindergarten cohort: eligible for the school lunch program, 1,996; 
not eligible for the school lunch program, 1,357. Grade 3 cohort: eligible for the school lunch program, 1,552; 
not eligible for the school lunch program, 434. Grade 6 cohort: eligible for the school lunch program, 900; 
not eligible for the school lunch program, 261. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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cumulative passing rate of 70 percent. In the kindergarten cohort both female and male 
English learner students had a final cumulative passing rate of 76 percent on the math 
content test. 


Figure C6. The difference in the final cumulative passing rates on the math content 
test for female and male English learner students was greatest for the grade 6 
cohort and least for the kindergarten cohort, 2006/07-2011/12 
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Note: Students’ English language proficiency assessment in 2006/07 served as a baseline, defining the initial 
population of English learner students for this analysis. In Utah students take grade-specific math content 
tests in elementary school and course-specific math content tests in secondary school. The number of English 
learner students in each subgroup is as follows. Kindergarten cohort: female, 1,632; male, 1,721. Grade 3 
cohort: female, 898; male, 1,088. Grade 6 cohort: female, 506; male, 655. 

Source: Authors' analysis of student-level data from the Utah State Office of Education, 2006/07-2011/12. 
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Notes 


1. Utah no longer administers these tests. The Utah Academic Language Proficiency 
Assessment has been replaced by WIDA ACCESS, and the subject matter content 
tests in English language arts and math have been replaced by the Student Assess- 
ment of Growth and Excellence. See box 2 and appendix A for more details. 

2. This report documents a study that was replicated in three states: Arizona, Nevada, 
and Utah. Thus, while the data and findings naturally differ from one report to the 
other, the explanation of why the study was done, the review of relevant literature, and 
the description of methods are nearly the same in each report. 

3. Before 2010/11 the Utah State Office of Education used both the Utah Academic 
Language Proficiency Assessment and the English language arts content test to iden- 
tify English learner students’ level of English language proficiency and to determine 
whether they would be reclassified as fluent English proficient; starting in 2010/11, only 
the Utah Academic Language Proficiency Assessment was used. 

4. The total number of cohort English proficiency level subgroups for which this study 
is measuring achievement gains (15 = 3 grade-level cohorts x 5 English language 
proficiency levels) differs from the total number of English proficiency progress sub- 
groups for which annual measurable achievement objective 1 progress is calculated 
(18 = 3 grade-level cohorts x 6 English proficiency progress levels below the reclassifi- 
cation level of bridging). Utah used seven levels of progress for determining whether 
an English learner student met the annual measurable achievement objective by sub- 
dividing beginning (level 2) into 2A and 2B and developing (level 3) into 3A and 3B 
(six levels from entering to expanding, plus the seventh level of bridging). However, of 
these 18 possible English proficiency progress subgroups, only 14 could be measured due 
to limitations in the study data. First, the study period was only six years or five years 
past the year 1 initial English language proficiency determination. As a result, there 
was not sufficient time to determine the total percentage of students at the lowest 
English proficiency level (three progress subgroups, one for each grade level cohort 
at the entering level in year 1) who would make the annual measurable achievement 
objective rate until reclassification (six steps or years). Second, Utah did not test stu- 
dents in the kindergarten cohort in the full set of reclassification tests until year 3 
of the study (second grade), so it was not possible to determine what percentage of 
kindergarten students at expanding proficiency, level 4 (one level below that needed to 
achieve reclassification as fluent English proficient) reclassified at the expected rate of 
one year (one progress subgroup, kindergarten English learner students at the expand- 
ing level in year 1). 

5. This study performed no statistical tests. Hence, the small differences that were found 
in this study may not yield a statistically significant difference. 
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